{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性回归分析与评估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1、导入必要的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np  # 矩阵操作\n",
    "import pandas as pd # SQL数据处理\n",
    "\n",
    "from sklearn.metrics import r2_score  #评价回归预测模型的性能\n",
    "\n",
    "import matplotlib.pyplot as plt   #画图\n",
    "import seaborn as sns\n",
    "\n",
    "# 图形出现在Notebook里而不是新窗口\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>holiday</th>\n",
       "      <th>workingday</th>\n",
       "      <th>yr</th>\n",
       "      <th>season_1</th>\n",
       "      <th>season_2</th>\n",
       "      <th>season_3</th>\n",
       "      <th>season_4</th>\n",
       "      <th>mnth_1</th>\n",
       "      <th>mnth_2</th>\n",
       "      <th>mnth_3</th>\n",
       "      <th>...</th>\n",
       "      <th>weekday_2</th>\n",
       "      <th>weekday_3</th>\n",
       "      <th>weekday_4</th>\n",
       "      <th>weekday_5</th>\n",
       "      <th>weekday_6</th>\n",
       "      <th>temp</th>\n",
       "      <th>atemp</th>\n",
       "      <th>hum</th>\n",
       "      <th>windspeed</th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355170</td>\n",
       "      <td>0.373517</td>\n",
       "      <td>0.828620</td>\n",
       "      <td>0.284606</td>\n",
       "      <td>0.110792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.379232</td>\n",
       "      <td>0.360541</td>\n",
       "      <td>0.715771</td>\n",
       "      <td>0.466215</td>\n",
       "      <td>0.089623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.171000</td>\n",
       "      <td>0.144830</td>\n",
       "      <td>0.449638</td>\n",
       "      <td>0.465740</td>\n",
       "      <td>0.152669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.175530</td>\n",
       "      <td>0.174649</td>\n",
       "      <td>0.607131</td>\n",
       "      <td>0.284297</td>\n",
       "      <td>0.177174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.209120</td>\n",
       "      <td>0.197158</td>\n",
       "      <td>0.449313</td>\n",
       "      <td>0.339143</td>\n",
       "      <td>0.181546</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 34 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   holiday  workingday  yr  season_1  season_2  season_3  season_4  mnth_1  \\\n",
       "0        0           0   0         1         0         0         0       1   \n",
       "1        0           0   0         1         0         0         0       1   \n",
       "2        0           1   0         1         0         0         0       1   \n",
       "3        0           1   0         1         0         0         0       1   \n",
       "4        0           1   0         1         0         0         0       1   \n",
       "\n",
       "   mnth_2  mnth_3  ...  weekday_2  weekday_3  weekday_4  weekday_5  weekday_6  \\\n",
       "0       0       0  ...          0          0          0          0          1   \n",
       "1       0       0  ...          0          0          0          0          0   \n",
       "2       0       0  ...          0          0          0          0          0   \n",
       "3       0       0  ...          1          0          0          0          0   \n",
       "4       0       0  ...          0          1          0          0          0   \n",
       "\n",
       "       temp     atemp       hum  windspeed       cnt  \n",
       "0  0.355170  0.373517  0.828620   0.284606  0.110792  \n",
       "1  0.379232  0.360541  0.715771   0.466215  0.089623  \n",
       "2  0.171000  0.144830  0.449638   0.465740  0.152669  \n",
       "3  0.175530  0.174649  0.607131   0.284297  0.177174  \n",
       "4  0.209120  0.197158  0.449313   0.339143  0.181546  \n",
       "\n",
       "[5 rows x 34 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"fe_day.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 从原始数据中分离输入特征x和输出y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df[\"cnt\"]\n",
    "\n",
    "X = df.drop([\"cnt\"], axis = 1)\n",
    "\n",
    "feat_names = X.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 将数据分割训练数据与测试数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(584, 33)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=33, test_size=0.2)\n",
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4、线性回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 最小二乘回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.1 最小二乘回归与预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>5.932996e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>5.932996e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>5.932996e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>5.932996e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>5.932996e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>7.918611e+11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>7.918611e+11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>7.918611e+11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>temp</td>\n",
       "      <td>2.856951e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>yr</td>\n",
       "      <td>2.199469e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>atemp</td>\n",
       "      <td>1.364158e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-1.405573e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>hum</td>\n",
       "      <td>-1.584221e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-4.615195e+10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>season_4</td>\n",
       "      <td>-4.036471e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_2</td>\n",
       "      <td>-4.036471e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-4.036471e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-4.036471e+12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>-1.632853e+13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-1.632853e+13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>workingday</td>\n",
       "      <td>-2.226152e+13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-2.226152e+13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns          coef\n",
       "25     weekday_3  5.932996e+12\n",
       "27     weekday_5  5.932996e+12\n",
       "26     weekday_4  5.932996e+12\n",
       "24     weekday_2  5.932996e+12\n",
       "23     weekday_1  5.932996e+12\n",
       "19  weathersit_1  7.918611e+11\n",
       "20  weathersit_2  7.918611e+11\n",
       "21  weathersit_3  7.918611e+11\n",
       "29          temp  2.856951e-01\n",
       "2             yr  2.199469e-01\n",
       "30         atemp  1.364158e-01\n",
       "32     windspeed -1.405573e-01\n",
       "31           hum -1.584221e-01\n",
       "15        mnth_9 -4.615195e+10\n",
       "11        mnth_5 -4.615195e+10\n",
       "12        mnth_6 -4.615195e+10\n",
       "14        mnth_8 -4.615195e+10\n",
       "9         mnth_3 -4.615195e+10\n",
       "16       mnth_10 -4.615195e+10\n",
       "10        mnth_4 -4.615195e+10\n",
       "13        mnth_7 -4.615195e+10\n",
       "8         mnth_2 -4.615195e+10\n",
       "7         mnth_1 -4.615195e+10\n",
       "17       mnth_11 -4.615195e+10\n",
       "18       mnth_12 -4.615195e+10\n",
       "6       season_4 -4.036471e+12\n",
       "4       season_2 -4.036471e+12\n",
       "5       season_3 -4.036471e+12\n",
       "3       season_1 -4.036471e+12\n",
       "28     weekday_6 -1.632853e+13\n",
       "22     weekday_0 -1.632853e+13\n",
       "1     workingday -2.226152e+13\n",
       "0        holiday -2.226152e+13"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "\n",
    "# 1.使用默认配置初始化学习器实例\n",
    "lr = LinearRegression()\n",
    "\n",
    "# 2.用训练数据训练模型参数\n",
    "lr.fit(X_train, y_train)\n",
    "\n",
    "# 3. 用训练好的模型对测试集进行预测\n",
    "y_test_pred_lr = lr.predict(X_test)\n",
    "y_train_pred_lr = lr.predict(X_train)\n",
    "\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names), \"coef\":list((lr.coef_.T))})\n",
    "fs.sort_values(by=['coef'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.2 最小二乘评价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最小二乘测试集R2得分 0.8184627716654066\n",
      "最小二乘训练集R2得分 0.8478413406411978\n"
     ]
    }
   ],
   "source": [
    "#测试集\n",
    "print('最小二乘测试集R2得分', r2_score(y_test, y_test_pred_lr))\n",
    "#训练集\n",
    "print('最小二乘训练集R2得分', r2_score(y_train, y_train_pred_lr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 岭回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.1岭回归与预测(R2 SCORE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_ridge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>yr</td>\n",
       "      <td>0.232385</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>temp</td>\n",
       "      <td>0.206065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>atemp</td>\n",
       "      <td>0.177930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>0.105384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>season_4</td>\n",
       "      <td>0.090386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>0.088549</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>0.044535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>0.040284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>0.038376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>0.029552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>0.026098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>0.024690</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>workingday</td>\n",
       "      <td>0.024348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>0.019057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_2</td>\n",
       "      <td>0.011460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>0.009518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>0.007005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>0.006973</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>0.001365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-0.003828</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-0.009499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-0.022139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-0.023626</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-0.026913</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-0.028307</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-0.036117</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-0.046737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-0.063298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-0.068807</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-0.092348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-0.124526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>hum</td>\n",
       "      <td>-0.130080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-0.149919</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns  coef_ridge\n",
       "2             yr    0.232385\n",
       "29          temp    0.206065\n",
       "30         atemp    0.177930\n",
       "19  weathersit_1    0.105384\n",
       "6       season_4    0.090386\n",
       "15        mnth_9    0.088549\n",
       "20  weathersit_2    0.044535\n",
       "12        mnth_6    0.040284\n",
       "11        mnth_5    0.038376\n",
       "14        mnth_8    0.029552\n",
       "28     weekday_6    0.026098\n",
       "16       mnth_10    0.024690\n",
       "1     workingday    0.024348\n",
       "9         mnth_3    0.019057\n",
       "4       season_2    0.011460\n",
       "27     weekday_5    0.009518\n",
       "25     weekday_3    0.007005\n",
       "26     weekday_4    0.006973\n",
       "10        mnth_4    0.001365\n",
       "24     weekday_2   -0.003828\n",
       "5       season_3   -0.009499\n",
       "22     weekday_0   -0.022139\n",
       "23     weekday_1   -0.023626\n",
       "13        mnth_7   -0.026913\n",
       "0        holiday   -0.028307\n",
       "8         mnth_2   -0.036117\n",
       "7         mnth_1   -0.046737\n",
       "17       mnth_11   -0.063298\n",
       "18       mnth_12   -0.068807\n",
       "3       season_1   -0.092348\n",
       "32     windspeed   -0.124526\n",
       "31           hum   -0.130080\n",
       "21  weathersit_3   -0.149919"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import  RidgeCV\n",
    "\n",
    "#1. 设置超参数（正则参数）范围\n",
    "alphas = [ 0.01, 0.1, 1, 10,100]\n",
    "\n",
    "#2. 生成一个RidgeCV实例\n",
    "ridge = RidgeCV(alphas=alphas, store_cv_values=True)  \n",
    "\n",
    "#3. 模型训练\n",
    "ridge.fit(X_train, y_train)    \n",
    "\n",
    "#4. 预测\n",
    "y_test_pred_ridge = ridge.predict(X_test)\n",
    "y_train_pred_ridge = ridge.predict(X_train)\n",
    "\n",
    "# 看看各特征的权重系数，系数的绝对值大小可视为该特征的重要性\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef_ridge\":list((ridge.coef_.T))})\n",
    "fs.sort_values(by=['coef_ridge'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.2 岭回归评价(R2 SCORE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "岭回归测试集R2得分 0.8292368308562362\n",
      "岭回归训练集R2得分 0.850231171960466\n"
     ]
    }
   ],
   "source": [
    "#测试集\n",
    "print('岭回归测试集R2得分', r2_score(y_test, y_test_pred_ridge))\n",
    "#训练集\n",
    "print('岭回归训练集R2得分', r2_score(y_train, y_train_pred_ridge))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Lasso回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3.1岭回归与预测(R2 SCORE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sklearn/model_selection/_split.py:1978: FutureWarning: The default value of cv will change from 3 to 5 in version 0.22. Specify it explicitly to silence this warning.\n",
      "  warnings.warn(CV_WARNING, FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns</th>\n",
       "      <th>coef_lasso</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>temp</td>\n",
       "      <td>0.310363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>yr</td>\n",
       "      <td>0.232175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>atemp</td>\n",
       "      <td>0.098630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>season_4</td>\n",
       "      <td>0.067214</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>mnth_9</td>\n",
       "      <td>0.066555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>weathersit_1</td>\n",
       "      <td>0.061779</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>mnth_10</td>\n",
       "      <td>0.025575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>mnth_5</td>\n",
       "      <td>0.023929</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>mnth_3</td>\n",
       "      <td>0.022012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>workingday</td>\n",
       "      <td>0.020358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>mnth_6</td>\n",
       "      <td>0.018832</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>weekday_6</td>\n",
       "      <td>0.014646</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>weekday_5</td>\n",
       "      <td>0.001806</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>season_2</td>\n",
       "      <td>0.000908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>mnth_4</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>mnth_8</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>weekday_4</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>weathersit_2</td>\n",
       "      <td>-0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>weekday_3</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>weekday_2</td>\n",
       "      <td>-0.005587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>season_3</td>\n",
       "      <td>-0.008176</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>mnth_2</td>\n",
       "      <td>-0.015498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>holiday</td>\n",
       "      <td>-0.020408</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>mnth_1</td>\n",
       "      <td>-0.025210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>weekday_1</td>\n",
       "      <td>-0.026095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>weekday_0</td>\n",
       "      <td>-0.026595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>mnth_11</td>\n",
       "      <td>-0.047862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>mnth_7</td>\n",
       "      <td>-0.049361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>mnth_12</td>\n",
       "      <td>-0.051377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>hum</td>\n",
       "      <td>-0.115536</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>windspeed</td>\n",
       "      <td>-0.116440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>season_1</td>\n",
       "      <td>-0.117204</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>weathersit_3</td>\n",
       "      <td>-0.190628</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         columns  coef_lasso\n",
       "29          temp    0.310363\n",
       "2             yr    0.232175\n",
       "30         atemp    0.098630\n",
       "6       season_4    0.067214\n",
       "15        mnth_9    0.066555\n",
       "19  weathersit_1    0.061779\n",
       "16       mnth_10    0.025575\n",
       "11        mnth_5    0.023929\n",
       "9         mnth_3    0.022012\n",
       "1     workingday    0.020358\n",
       "12        mnth_6    0.018832\n",
       "28     weekday_6    0.014646\n",
       "27     weekday_5    0.001806\n",
       "4       season_2    0.000908\n",
       "10        mnth_4   -0.000000\n",
       "14        mnth_8    0.000000\n",
       "26     weekday_4    0.000000\n",
       "20  weathersit_2   -0.000000\n",
       "25     weekday_3    0.000000\n",
       "24     weekday_2   -0.005587\n",
       "5       season_3   -0.008176\n",
       "8         mnth_2   -0.015498\n",
       "0        holiday   -0.020408\n",
       "7         mnth_1   -0.025210\n",
       "23     weekday_1   -0.026095\n",
       "22     weekday_0   -0.026595\n",
       "17       mnth_11   -0.047862\n",
       "13        mnth_7   -0.049361\n",
       "18       mnth_12   -0.051377\n",
       "31           hum   -0.115536\n",
       "32     windspeed   -0.116440\n",
       "3       season_1   -0.117204\n",
       "21  weathersit_3   -0.190628"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LassoCV\n",
    "\n",
    "lasso = LassoCV()  \n",
    "\n",
    "lasso.fit(X_train, y_train)  \n",
    "\n",
    "y_test_pred_lasso = lasso.predict(X_test)\n",
    "y_train_pred_lasso = lasso.predict(X_train)\n",
    "\n",
    "fs = pd.DataFrame({\"columns\":list(feat_names),\"coef_lasso\":list((lasso.coef_.T))})\n",
    "fs.sort_values(by=['coef_lasso'],ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3.2 Lasso回归评价(R2 SCORE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lasso测试集R2得分 0.833272444276309\n",
      "Lasso训练集R2得分 0.8486048662748503\n"
     ]
    }
   ],
   "source": [
    "#测试集\n",
    "print('Lasso测试集R2得分', r2_score(y_test, y_test_pred_lasso))\n",
    "#训练集\n",
    "print('Lasso训练集R2得分', r2_score(y_train, y_train_pred_lasso))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5、作业"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 对连续型特征，可以用哪个函数可视化其分布？（给出你最常用的一个即可），并根据代码运行结果给出示例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以使用hist和displot函数可视化分布直方图，“数据分析”作业中有相关代码。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2 对两个连续型特征，可以用哪个函数得到这两个特征之间的相关性？根据代码运行结果，给出示例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "corr函数可以分析两两连续特征的相关性。“数据分析”作业中有相关代码。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3 如果发现特征之间有较强的相关性，在选择线性回归模型时应该采取什么措施。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "较强相关性的特征可以进行PCA降维和加正则。（怎么降维不知道啊）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.4 当采用带正则的模型以及采用随机梯度下降优化算法时，需要对输入（连续型）特征进行去量纲预处理。课程代码给出了用标准化（StandardScaler）的结果，请改成最小最大缩放（MinMaxScaler）去量纲 ，并重新训练最小二乘线性回归、岭回归、和Lasso模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "“特征工程”作业中有相关代码"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5 代码中给出了岭回归（RidgeCV）和Lasso（LassoCV）超参数（alpha_）调优的过程，请结合两个最佳模型以及最小二乘线性回归模型的结果，给出什么场合应该用岭回归，什么场合用Lasso，什么场合用最小二乘。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最小二乘回归：当模型比较简单，模型在训练样本和测试样本上都能有比较好的拟合就不需要加入正则，这时候就可以选择最小二乘回归。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "岭回归：岭回归可以使线性回归系数收缩，可以使模型更稳定。当输入特征之间有比较强的相关性时就可以选择岭回归。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lasso：Lasso也会收缩回归系数，并且在正则参数取合适值时可以得到稀疏模型。当输入特征较多，有些特征与目标的相关性较弱时就可以选择Lasso。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
